Generalization Ability of Frequency Information Processing Neural Networks

Frequency domain generalization in complex valued neural networks is analyzed The complex valued neu ral networks consist of variable delay lines neural connection conductance and complex neuron nonlinearity The learning of frequency pro les is realized by adjusting the delay time and the conductance using backprop agation process The information geometry is discussed for obtaining a parameter region where a reasonable generalization is realized in frequency space It is found that there are error function minima periodically both in delay time domain and input signal frequency domain Experiments demonstrate that a stable learn ing and a reasonable generalization in the frequency domain are realized in a parameter range suggested by the theory This result is applied not only to direct frequency signal processing but also to future optical computing and quantum neural devices Introduction In the years since a pioneering work a lot of ideas and experiments on complex valued neural networks have been reported Especially in these years the complex valued neural networks are supposed to create new elds associated with optical computing and quantum neural devices The most speci c feature of general neural networks lies in the distributed and parallel construction When we construct highly parallel neural systems in the future we will have to treat quantum aspects of the information carrier to realize smaller and less power consumptive devices no matter what kind of information carrier we will use The complex valued neural networks can deal with the phase information which is the origin of coherent aspects of quantum phenomena However it is also important to obtain a direct relation between the frequency of the information carrier and the behavior of the neural networks When we modulate the network behavior by changing potential in a ballistic or coherent electronic neural devices for example it is necessary to have the knowledge on the in uence of carrier energy equivalent to the frequency on the network behavior Recently the direct frequency information processing using complex valued neural networks has been proposed and the learning process has also been demonstrated In this paper the information geometry is analyzed and discussed by introducing a local error function by which the learning process can be investigated It is found that there are error function minima periodically both in the delay time domain and the input signal frequency domain Experiments demonstrate that a stable learning and a reasonable generalization in the frequency domain are realized in a parameter range suggested by the theory This result is applied not only to direct frequency signal processing but also to future optical computing and quantum neural devices Information geometry in delay time and frequency domains Figure shows the basic construction of the complex valued neural networks for frequency information pro cessing An input signal electromagnetic wave for example is fed to the input terminals and processed in the neural network The output signals are detected by using phase sensitive detectors with a phase reference provided by the same input signal Therefore the neural system forms totally a so called self homodyne circuit although only the neural network part is shown in Fig The activation function of the complex valued neural network is determined as f sk exp i k A tanh sk m exp i k sk exp i k N X